1 link tagged with all of: prompt engineering + reliability + evaluation + vedder
Click any tag below to further narrow down your results
Links
Spotify built Vedder, an AI data assistant that uses domain-specific “clusters” curated by expert teams to provide reliable SQL-generated insights. Each cluster combines relevant datasets, vetted question–SQL pairs, and docs, with automated health scoring and feedback loops to keep context accurate and scalable.
- Auto-generating question-SQL examples from query logs mostly failed—only 12.5% passed expert review, proving curated human examples beat scraped ones
- Vedder's reliability comes from domain-expert-curated "clusters" (tables + vetted SQL examples + docs), not a bigger LLM
- Health scoring tracks schema drift and broken examples so experts know which clusters need fixing as data evolves
- Adopted by 2,100+ employees for 13,000+ conversations since August 2025, showing real traction beyond a pilot